DOCUMENT RESUME 



ED 064 599 

AUTHOR 

TITLE 

INSTITUTION 

REPORT NO 
PUB DATE 
NOTE 

EDRS PRICE 
DESCRIPTORS 



AC 012 691 

Spautz, Michael E. 

A Survey of *-he Effectiveness of Management 
Development Programs. 

Civil service Commission. Washington, D.C. Bureau of 
Training. 

Pamp-T-1 4 

May 71 

68p. 

MF-S0.65 HC-S3.29 

Business; Course Evaluation; Industry; *Management 
Development; *Management Education; *Program 
Evaluation; Questionnaires; Research Criteria- 
Research Methodology; ♦Surveys * 

ABSTRACT 

„ ... . T 5 is P a P er summarizes the results of a survey of 

^^, t =“ PUbliShed t . reSearch studies » conducted in private industry 
f" d . bus ^ ness ' on the effectiveness of a variety of programs designed 

obtain^nfo™^ 961 *^ abili ^ ies * The purpose of the survey was to 
obtain information about private industry training evaluation 

practices which might be used or adapted for use by Agencies. 

? * otad survey sample of 244, 17 studies were located that 

are or^i^ed 1 ii«-^T J9h b ® included in the report. The summaries 

of crite^ia acrai nSt whi ^ cate 9 ories defined in terms of the type 
oriterfe? and*i2?^n?l££«i' the programs are evaluated: (1) external 
f„ (2) internal criteria, in addition, the criteria used 

study ar ? classified into four levels of 
sophistication: (1) participants* reactions; (2) learning; (3) 

rr< nmmfr i?f^ P9e; “a results. Some general conclusions, 
pres^ted? (ClT' a “ d 016 questionnaire used in the study are also 



ED 064599 



U S. DEPARTMENT OF HEALTH, 
EDUCATION A WELFARE 
OFFICE OF EDUCATION 
THIS DOCUMENT HAS BEEN REPRO- 
DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIG- 
INATING IT POINTS OF VIEW OR OPIN 
IONS STATED DO NOT NECESSARILY 
REPRESENT OFFICIAL OFFICE OF EDU- 
CATION POSITION OR POLICY 



A Survey of the Effectiveness of 
Management Development Programs 



C3- 

v3 



U S. CIVIL SERVICE COMMISSION 

BUREAU OF TRAINING 



ED 064599 



Pamphlet T-14 



A SURVEY OF THE EFFECTIVENESS 
OF 

MANAGEMENT DEVELOPMENT PROGRAMS 



May 1971 



U. S. Civil Service Commission 
Bureau of Training 




• t 



2 



Foreword 



In this report Dr. Michael E. Spautz summarizes the results of a ,‘iurvey he conducted 
during the summer of 1970. Dr. Spautz's efforts were sponsored by the "’'•omission's 
San Francisco Regional Office as a part of the summer faculty program. 

The objective of the survey was to find out what private industry is doing to 
validate management development programs. The central office found the report to be in- 
formative and useful and is reprinting it here with the thought that others will find it 
equally so. 
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A Survey of the Effectiveness 
of Management Development Programs 
Introduction 



This paper summarizes the results of a survey of recent unpublished research studies, 
conducted in private Industry and business, on the effectiveness of a variety of programs 
designed to develop management abilities. 

Objective 

The purpose of the survey, which was sponsored by the U.S. Civil Service Coamission, was 
"to obtain information about private Industry training evaluation practices which 
be used or adapted for use by Federal agencies." 

This report merely sumsarizes the survey data and draws a few implications. It is not 
within the scope of this report to go deeply into theoretical issues, the design of manage- 
ment development programs, or the methodology of training research. These are quite well 

treated in several recent publications by Campbell, et al. CD*. Hesaeling (2). House 
(3), and Kirkpatrick (4). 



In view of the limited time and resources available, it was decided to confine the survey 
to the state of California, where business and industry are highly concentrated in a few 
metropolitan areas. In some cases, while the respondent was presently located in California, 
the research was originally done in organizations in other states. (It is estimated that 
the resulting information probably represents about 10 to 20% of such unpublished research 
results available in the entire United States.) it was further decided to emphasize research 
on the lower levels of white collar supervision and middle management. For this reason, 
several promising leads to studies of executive-level programs were not followed up. 



Method 



Two hundred and ten ahort 



questionnaires (see Appendix 1) were sent to management develop- 



ment practitioners and researchers, whose names were selected from membership directories 
of the American Society for Training and Development, and the American Psychological 



^Numbers in parentheses denote a reference 



to be found on page 69. 



Association. The return rate was 68X, and of these, 40 respondents indicated that 
they had recently been involved in unpublished research on the effectiveness of manage- 
ment development. Through these initial contacts an additional 34 leads were generated, 
bringing the total survey sample to 244. The following summaries are based on information 
obtained from numerous subsequent face-to-face interviews, telephone interviews, and 
unpublished material furnished by the respondents. In all cases, a preliminary draft 
of a specific research summary was sent to the respondent for correction and approval. 

It should be noted that there was no attempt to randomize the survey sample, as there was 
no interest in measuring the proportion of individuals or organizations that have been 
engaged in such research. Rather, names were selected on the basis of high probability of 
payoff, i,e., individuals known to be research-oriented and/or working for relatively 
large organizations, which could be assumed to possess the necessary financial and 
managerial resources to support sizeable management development and related research 
programs . 

Results 

Seventeen unpublished research studies were located that seemed, to the present reviewer, 
to be substantial enough for inclusion in this report. For some of them, there was 
adequate information to warrant an extensive, standardized summary (see Appendix 2 for a 
skeleton outline); while for others, only fragmentary information was available. Studies 
that focus exclusively on participant reactions are excluded altogether from this report, 
as being of marginal interest. Additional information about management development obtained 
from personal interviews with other respondents, is presented later, in the discussion 
section. 

The various summaries are organized into the same two major categories as used by Campbell, 
et al., (1, pp. 287 ff). These categories ere defined in terms of the type of criteria 
against which the programs are evaluated, namely: 1) "external , ' criteria (e.g,, objective 

performance measures on the job, operating data such as turnover and grievances, and 
performance ratings by supervisors, peers and subordinates); 2) ’•internal" criteria 
(e.g., knowledge and attitude tests, questionnaires completed by the participants to get 
their opinions about the program). 
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Within each major category, the criteria used in a particular study will be further 
classified according to Kirkpatrick's system (4). He recommends classifying evaluation 
methods into four levels of sophistication, namely; U participants' reactions; 2) learn- 
ing; 3) behavior change and 4) results. Certain approaches, such as attitude test 
scores, opinions of a man's colleagues about his subsequent behavior, morale and job 
satisfaction measures, and the impressions of top management concerning the impact of a 
program on the organisational climate, are difficult to classify according to Kirkpatrick's 

system; no attempt will be made to force the following research approaches into predeter, 
mined categories where they don» t obviously fit. 

An attempt was made to categorise the studies in terms of specific criteria and training 

s, but, as Campbell, et al., (1) found, this proved to be unworkable because most 
studies were too complex, involving several criteria or several methods. 
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Discussion 



The chronicle to follow may leave one with the impression that management development 

invariably pays off. A more realistic view might be that, in view of the strong tendency 

to suppress negative results, we have here only some suggectlve evidence about the validity 

of selected programs in particular situations, with certain kinds of subjects, contents 

and methods, as gauged against certain criteria. To generalise unequivocally about -the 

effectiveness of management development- at large would thus not be warranted by the data 
at hand. 



It might appear from these studies that external criteria are more frequently employed 
than internal criteria. Once again, such an inference would be erroneous, for similar 
reasons that less worthy efforts are less likely to be followed through to completion 
and subsequently reported, whether in the published literature or in a survey such a. this. 

While some of these studies are as sophisticated and conclusive as some in the published 
literature, most are of limited value from a scientific point of view, two of the most 
co»o„ shortcomings .re the failure to use control groups, and the failure to replicate 
findings. Perhaps the main contribution of this survey is to give the reader an Increased 
awareness of the variety of training approaches and research methodologies that have been 
tried out and perhaps a few tip. on pitfalls to be avoided in his own programs. 

Interviews with training executives, trainers and researchers revealed a widespread 
agreement as to the difficulty of doing conclusive research in this field. Research on 
assessment programs (e.g. , test validation studies) is generally acknowledged to be 
relatively easy, because the ..predictors- are most readily identified, quantified and 
controlled, and more amenable to statistical treatment. One of the main problem areas 
in assessment research, namely -the criterion problem , is at least as troublesome to 
developmental researchers. Until more reliable and meaningful ultimate criteria of 

effectiveness are available, we may have to be content with -weak inferences- about the 
validity of many of our human resources programs. 
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In short, a large proportion of interviews, including some represented in the forthcoming 
summaries, have expressed skepticism about the feasibility of "proving*' the effectiveness 
of a specific training technique, such as case analysis, role playing, group-discussion, 
or simulation exercises. Similarly, few expect to ever "prove" that the Managerial Grid, 
or Likert’s System 4, or McGregor's Theory V, etc,, pays off in terms of increased profits 
or employee satisfaction. In view of the inevitable presence of uncontrolled factors, 
such as changes in the level of the economy, shifting patterns of competition and con- 
sumer demands, and the "Hawthorne effect", most professionals appear to be reconciled 
to something less than the kind of "evidence" required to conclusively establish the 
validity and practical value of other management practices, marketing and engineering, 
for example. Offsetting this apparent despair of ever turning management development into 
a hard science, however, is an equally widespread faith and hope, based on a rapidly 
accumulating storehouse of circumstantial evidence, that, like education, management 
development programs in general , do appear to pay off in the long run. While widespread, 
this skepticism is not universal, as here and there one can find purists and dissenters 
who feel that pin-point research is indeed possible, if we are only sufficiently dedicated 
and sophisticated in our measurement techniques. Promising suggestions, not yet widely 
attempted, include the use of moderator variables to categorize trainees into more 
homogeneous samples, for research on differential effects. For example, there is reason 
to suspect that certain character (or personality, or attitude) dimensions such as 
authoritarianism, need for achievement, perceived internal - or external - control of 
rewards, may have an interactive effect with a particular training outcome. Just as a 
highly intelligent student is likely to learn more, faster and better than a less 
intelligent student, so a non-authoritarian manager may be more likely to profit from 
a participative developmental experience. However, this is the kind of hypothesis that 
apparently is seldom tested, perhaps because it is difficult to get support for research 
that may be seen to have serious political implications. 

However, an interesting but paradoxical example of the interaction of personality 
characteristics with performance, was recently reported by Perczei. (5) Essentially, 
what Perczel found was that ability and motivation in managers, while positively 



correlated at high and at low levels of performance, are negatively correlated at 
asderate levels of performance, while his study was not done in a developmental context, 
his results may have important implications for management development. For example, 
consider a typical program, whose subjects are mostly moderate performers, not particularly 
high or low. m theory, the brighter ones, after training, should be better .performers 
that the duller ones, since, being brighter, they could be expected to learn more. 

However, given PercseUs findings, this result would be likely only if specific steps 
are taken to increase the relatively low level of motivation of the brighter students, 
who have apparently learned to Coast. It is for this reason that some development experts 
feel that the key to increasing productivity in-industry is not to increase the motivation 
of everyone on the payroll but to concentrate on increasing the motivation of under- 
achievers. White-collar underachievement may be in part a function of social pressure 
against -rate busters... and in part a function of -following the lines of least resistance- 
in an, affluent society where the Protestant ethic appears to be obsolescent. 

in following up on the success of graduates of a T & D program it is easy to overlook other 
factors that can trap us into making erroneous conclusions. 

For example, Rusmore (b, found differential promotion rates associated with various 
position description factors, in two large public service organisations. In other words, 
if traditional measures of success, in terms of promotion rate are used to evaluate the 
effectiveness of management development, for a heterogeneous gtoup of managers, it will 
be desirable to stratify the trainees in terms of their organisational functions. Rusmorfs 
evidence shows that managers have a relatively high probability of success U.e., promotion 
rate, if their main functions are in long range planning, the xerclse of broad powers 

and authority, or in utilising technical knowledge of products and markets. Conversely, 

those whose main function is to provide staff se ru<P»c 

P viae staff services to non-operating units (e.g. f 

Personnel?,, have a relatively low probability of success. 

If such factors are not considered in evaluating the impact of development programs on a 
man.s career, for example, it is likely that a man with high potential who happens to be 
stuck in a low- potential Job. cop.d be erroneously labeled as a failure, even if hls own 



promotion rate is superior to the mean of his functional peers. In such a case* if his 
progress the organization appears to fall short of the progress of more fortunately 
placed managers, we would be in danger of labeling him as a failure, and erroneously 
concluding that, for him, a development program (e.g. , an MBA education) had no measur- 
able effect. 

And conversely, a man who is promoted ahead of his age group may be considered more 
successful and, therefore, more talented. Such an Inference might be in error, as the 
promotion rate could be more associated with the tasks of the position than with the 
man's qualifications. 

The advisability of replicat. , validity studies is demonstrated by the following 
experience of the present author. In 1962, an attempt was made to measure changes in 
attitudes associated with sensitivity training of business executives at U.C.L.A. 

About sixty trainees participated in 4 T-groups of about 15 each. (The program is 
described elsewhere in this report.) The attitude change instrument used was a specially 
designed version of the Semantic Differential, using only the evaluative dimension. 

At the beginning, and again at the end, of the program, each trainee completed the form 
to indicate his attitudes toward several key concepts, namely: Me, as I am; Me, as I 

would like to be; My Boss; My Colleagues; My Job. Analysis of the results indicated 
significant shifts in attitudes in the predicted direction, showing attitude closure 
between the first two concepts (= more acceptance of the self), and Improved attitudes 
toward some of the other concepts. These results were interpreted as support for the 
effectiveness of the T-group experience. 

However, it was decided to replicate the study with minor changes, during the next round 
of T-groups, with another sample of about 60. This time there were no statistically 
significant improvements in their attitudes. In fact, in one of the T-groups, a distinct 
change for the worse was apparent, which upon investigation was attributed to some sort of 
unfortunate episode that occurred in that group right before the instrument was administered 
at the last meeting. (Because of the confidentiality of the T-group situation, the nature 
of the unfortunate episode was not revealed.) In any case, the results of the replication 
O -8- 
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failed to substantiate the results obtained earlier. The fact that the study was not 
submitted for publication (the fault of the present author) tends to support the widespread 
suspicion of bias in the published literature. One can only speculate as to how the 
poiitive results reported in this survey report would stand up under replication! 

The use of sensitivity training for management development may be entering a new phase, 
as many of the present survey respondents seem to agree. Training executives more often 
than not indicated that they are moving to more structured, problem-oriented forms of 
laboratory training. For example, consider the following information about the trend 
in one of the foremost users of laboratory training, 

A long-time leader in the use of experience -based learning and group development methods 
for management and executive development, TRW Systems of Redondo Beach, California, 
has over the past several years moved aggressively into Organization Development. As the 
following excerpt from N.I.C.B. Case Study 10 shows, this 0D movement represents not only 
an emphasis on individual development, but also a team-building, group problem-solving, 
management -for- re suits approach. 

"TRW began experimenting with one of the elements of the process - that of 
sensitivity training - in 1963, and for quite some time the company's involvement 
included sending a few executives at a time to an NTL or a university laboratory. 

While the company regarded these "classic" laboratory training sessions as 
valuable for the men who attended them, there was no "built-in" way of linking 
the experience into their behavior back on the job. The vice president for 
industrial relations said, "The laboratory often is an intensive experience, but 
people can experience tremendous letdowns when they return to their work culture, 
even if that culture is a highly supportive one." 

Task-oriented T- Group 

For this reason, TRW has modified sensitivity training to include job - or task- 
oriented development. "If people are in a lab and they learn to level with each 
other about their feelings, they have begun to do something useful," said the same 
vice president. "But if that's ail they do, the experience has relatively minimal 
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usefulness compared to what it could accomplish if people relate to each other within 
the organizational setting around task issues." (7) 

This does not mean that TRW is phasing out of sensitivity training, but that it 
has added, with a significant investment of energy, other formats such as team 
building and intergroup problem solving meetings to focus less on strictly personal 
and interpersonal behavior, more on problem identification and solving, communications, 
goals and objectives of the organization. In short, there is increased emphasis 
on "team building" and problem solving. 

As an integrated part of the total management development program, TRW has also 
moved toward a greater use of Management Training - individually focused, content 
oriented training in managerial skills and knowledge. A major part of this training 
program is a nine day, in- house Basic Management Development Course for middle 
managers. 
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STUDIES USING 
EXTERNAL CRITERIA 



(Note that several of these studies also use internal 
criteria. Also, note that parts of these summaries 
are verbatim quotes from original unpublished source 
documents, while other parts are paraphrases. In all 
cases, these summaries were prepared by the present 
author, but cleared by the survey respondent.) 



1. Bank of America, San Francisco, Ca. 

2. Supervisory Training Conference (Operations) 

a. Objectives: 

1) To give useful information and practice in the supervisory skills required 
of operations officers* 

2) To clarify the leadership Requirements and opportunities of first- line 
supervisors, 

3) Specific behavioral changes sought include, but are not confined to, improve- 
ments in the following; method of employee coaching and counseling; open-ended 
interviewing, cross-training in spare time; setting high standards while 
showing respect for the individual; using specific incidents in performance 
appraisal; recruiting employees; using telephone checks in hiring. 

b. Participants numbered over 400 branch operations officers, who supervise the 
work of. tellers, clerks and office machine operators. About two-thirds were 
men, ranging in age from the low 20’ s to low 60' s, averaging about 12 years 
with the bank. 

c. Company training facilities were used. 

d. The program lasts three days, and is offered quarterly. 

e. Trainers are former successful operations officers, trained in teaching methods. 

f. Subject matter includes; the supervisor’s job; interviewing and selecting new 
employees; training methods and motivating employees; counseling and coaching; 
planning; performance reports; and the responsibilities and functions of Area 
Administration. Extensive participation is used, including case analyses, 
incident process, role playing, and oral presentations with feedback. 

3. Evaluation 

a. Several types: reactions of participants (not reported here, but stated to be 

uniformly favorable); learning; behavioral change; and results. 

b. A sample of 100 participants was randomly selected, stratified by size and 
location. Evaluation information was obtained directly from the participants, 
from their immediate superiors (branch managers), and from one of the participant’s 
subordinates, who was selected by the branch manager. 
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Instruments used were: 



1) A 15-item Job Information Survey (a behavioral ly-oriented knowledge test), 

was filled out by each participant on the basis of self -perceived current 
(one year after training) behavior, and also on the basis of how he thought 
he would have answered the question a year earlier (pre- training). The 
questions were designed to cover the job functions of: employee selection, 

orientation, training, staff meetings, coaching and counseling, and writing 
performance reports. 

2) A short survey form, the Branch Manager Questionnaire, was completed by the 
participants* superiors; it was designed to provide job behavior information 
comparable to that provided by the participant. 

3) A longer, similar survey form to be completed by one of the participant's 
subordinates. 

4) A summary of branch operation statistics, measuring employee absenteeism and 
vacations, "current index" (a complex, quantitative measure of the bank's 
productivity) and employee separations. Data from a period of three months 
right nefore training, and again for a three month period after training, 
were used, to see if there were any measurable changes. 

Results: 



1) On the Job Information Survey, filled out by the participants, the following 
changes were observed: 



Average test score before 
Average test score after 
Average test score improvement 



Male Female 

(t^SO) (N* 30 ) 

38.6 39.2 

58.4 56.4 

19.8 17.2 



To suumarize, substantial improvement was shown by both groups but inter-group 
differences were not significant. 



Item 2 of the 15-item test will serve as an example of the method used to 
measure results: 



No. of replies 
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One year 
Ago 

(pre- training) 



Present 

Answer 



Difference 



23 



When 1 must reprimand a teller 
for excessive tardiness in the 
morning, I: 

A. Say good-naturedly, "Good 
afternoon?* 

B, Tell her if she continues 
to be late it will affect 
her next salary increase 



C. Remind her that the other girls 

aren’t late and she is upsetting 
morale. 16 

D. Ask her why she is late and what 

specifically she is going to do 
about it. 35 



-21 



74 



-12 



+39 



2) On tne Branch Manager Questi onna ire , the following mean ratings were given 
to the participants by their superiors (note that the ratings were made on 
scales ranging from 1 s very good, to 5 = poor): 



Area Rated 


Before 


After 


Change 


Selection 


2.5 


2.0 


.5 


Orientation 


2.5 


2.0 


.5 


Training 


2.6 


2.1 


.5 


Staff meetings 


2.5 


1.9 


.6 


Coaching 


2.8 


2.2 


.6 


Performance appraisals 


2.5 


1.9 


.6 



Statistical treatment of these data indicated that each change was significant, 
revealing a consistent pattern of improvement. (Management feels that the 
real amount of improvement is obscured by the perhaps artifically high 
•’before" ratings, which are "above average".) 

3) The Subordinate Questionnaire data were not analyzed, as their usefulness 
was undermined by excessive use of the "don’t know" category and blank 

-15- 
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responses. Interpretation of th^ results was also hampered by the absence 
of before- training data. 

4) on the branch operating statistics criteria, the observed changes were not 
considered significant. This result was anticipated, because of the short 
duration of the training program, uncontrolled factors, and because the data 
were too far removed from the training objectives to provide suitable criteria 

of behavioral change. 

4. Conclusions 

The above results provide evidence of the effectiveness of the training program, on 
the basis of self-reported behavioral change, and perceptions of the participants' 
superiors. However, the results are based on an incomplete analysis of the data, 
a*»d additional analyses (e.g. , breaking it down by branch size and location) are 
contemplated. In view of the post-hoc design of this study, the researchers caution 
that the results, while generally positive, do not conclusively demonstrate the 
program's effectiveness. 
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Hughes Aircraft Co., Aerospace Group, Culver City, CA. 

Management Action Workshop 

a. Objectives: 

1) Learn new ways of understanding how organizations function. 

2) Develop more effective skills in communication with others, including 
such day-to-day needs as employment interviewing, performance appraisal, 
and disciplining. 

3) Acquire the capacity to more effectively delegate, motivate, and set 
objectives with subordinates. 

4) Identify one's own leadership styles and areas of potentially greater 
effectiveness. 

5) Improve decision-making ability. 

6) Better understand and more effectively interface with the various company 
management systems. 

7) Satisfy contractor requirements for greater management effectiveness. 

8) Feel a more significant part of management. 

b. The participants are middle level managers, including Section Heads, Senior 
Project Engineers, Senior Systems Engineers, Chiefs and Superintendents. As of 
this date, 467 participants have gone through 27 workshops. 

c. The setting was at a suburban hotel, in order to remove the participants from 
day-to-day problems at home and at work. 

d* The workshops lasted 4 days, full time, from Tuesday evening to Saturday 
afternoon. The participants stayed at the hotel overnight. 

e. There were six workshop staff members who participated in various parts of the 
workshop including one full time and four part time/ consultant members of the 
Personnel Development and Organizational Effectiveness Department and the 
Assistant Manager of the Management Systems Division. 

f. The program is a workshop design: theory presentations followed by exercises 

which will simulate key job activities in order to emphasize application to the 
job. Appropriate time is provided to permit discussion of important issues 
facing managers. 



Each workshop is limited to 20 participants to permit a high degree of 
personal involvement and interaction. 



The workshop is divided into five major content areas: 

1) Conmunicatlons 

2) The Man in the HAC Management System 

3) Job Performance (including setting objectives, delegation, motivation 
and performance appraisal). 

4) Leadership Styles and Leadership Effectiveness. 

3) Decision-making and Problem-solving. 

The main focus of these sections is on a real-life, back home problem. One 
section of the workshop is devoted to identifying on-the-job problems which 
are to be worked on using workshop skills. 

Finally, about two months after the workshop, the participants reconvene for 
a brief review of the workshop and an opportunity to discuss the results of 
the back-home problems. Participants are thus able to share ideas (based 
on actual experiences) about the effectiveness of the various approaches 
presented during the workshop. 

3. Evaluation 

a. Several types of evaluation were made. Including: 1) participant reactions, 

2) attitude changes, 3) knowledge ami skill improvement, 4) subsequent 
problem-solving, 5) cost effectiveness. 

b. Methods: 

1) Participant reactions were obtained by means of oral and written critiques, 
using a standard set of open-ended questions to get at various aspects of the 
Workshop, including general reactions, the extent to which his expectations 
were met, how it could have been more relevant to his job, and the value 
of the various workshop experiences. 
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2) Attitude changes were measured by means of a special semantic differential 
questionnaire given before, immediately after and two months after the 
workshop. This questionnaire uses 16 bipolar phrases to measure each 
participant's expectation (pre-test) and, later, actual views (post-test) 
of the total Workshop experience. A quantitative scoring system was 
developed to summarize the "expectations'* and "actual" views for each of 
the 27 workshops, and profile charts were drawn. 

3) Ferceived changes In the participants knowledge and aknu were measured bv 
weans of ratings on six functions, each which has a maximum possible rating 
of 100 (see the results below, for the six functions). Each participant and 
his immediate superior Independently completed the rating before the Workshop, 

and again four months after the Workshop, estimating the participant- s 
competence. 

4) Subsequent problem-solving was determined by mean, of the participant's 

self-reported success In solving a real-life on-the-job problem that he had 

Identified before the Workshop. This Information was obtained Informally at 

special group meetings with the participants, held two months after the 
Workshop. 

5) Cost effectiveness was likewise informally determined, as objective data were 

not available. A small sample of eight participants was Interviewed and 

asked whether participation In the Workshops had increased cost effectiveness, 
c. Results: 

1) The reactions of the participants were generally very favorable, with few 
reservations. 

2) On the semantic differentials, average scores for the 27 groups were as 
follows, on a scale having a maximum high value of 100: 





Expectations 


Actual 


Gain 


High 


81 


87 


20 


Average 


73 


81 


g 


Low 


67 


*69 


-i 



*The workshop administrator was absent from this session An H 
in for him cni 8e881 on and other staff members filled 



O 
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This result was interpreted to mean that all groups but one found the workshop 
to surpass their expectations- -which is in keeping with the abov f j findings 
on their reactions, 

3) On the rating of estimated corape tence, the average scores are as follows; 

Ratings by Self-Ratings 

Supervisors by Participan ts 





Before 


After 


Gain 


Before 


After 


Gain 


How companies work 


65 


74 


9 


57 


66 


9 


How to be a good mgr. 


76 


82 


6 


70 


77 


7 


Where AG* is going 


63 


74 


11 


62 


64 


2 


How does AG* work 


68 


75 


7 


61 


68 


7 


How leaders lead 


63 


77 


14 


63 


72 


9 


How people talk to 
each other 


76 


86 


10 


74 


80 


6 


How people work together 


78 


88 


10 


75 


80 


5 



(*AG= Hughes Aerospace 
Group) 



(Note: on the scales used, a score below 50 Jndicates a negative rating; 

the maximum possible score is 100.) 

4) On the cost effectiveness variable, ail eight responses were in the affirmative 
although only half of them could be specific. Tha specific illustrations were 
e.s follows: From what I learned about motivation, we taught a group of TEs 

to use computers. Now, they contribute so much more it's 
unbe lie veable... increased efficiency plus doing jobs that more 
expensive engineers did, 

...a smoother running organization. Have been able to channel 
energies of ry people into more productive pursuits. 

Sure, it's saved money... met experts at the Workshop that don't 
cost... exposed ma to other specialist areas to consult with. 

Made my job easier. 
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...an opportunity to sit back and start better organizing my workload. Now able 
to be more flexible and act quickly. .. learned to carry the job to the people - to delegate. 
I found free time for more important work. 

4. Conclusions: 

"The above results are gratifying give con' Iderable credence to the hypothesis 
that attendance at the Workshops leads to increased managerial effectiveness. 

Although some participants apparently do not benefit from the experience, we estimate 
that 80-90X do benefit in a measurable way. In terms of the total Aerospace Group, 
the Management Action Workshop participants can be viewed as a series of small 
payoffs which collectively integrate into a measurable improvement in Aerospace 
managerial effectiveness. Continuous management development activity is 
necessary in order to continue the trend in that direction." 
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Kaiser Aluminum & Chemical Corp., Oakland, Ca. 

Organization Development laboratory 

a. Objectives; 

1) To build a more effective team, with concentration on communication 

2) To improve group decision-making skills 

3) To secure commitment to meaningful goals 

b. The participants were 11 managers from the various levels and functional 
specialities of a subsidiary plant. These men represented the core of the 
management team, so that they might be said to constitute a “family group'*, 
as opposed to a gathering of unrelated managers typical of many other T & D 
programs. 

c. The sessions took place on Company premises. 

d. The laboratory experiences covered a % day period, full time, preceded by 
several preliminary meetings designed to work out the general objectives of the 
developmental sessions proper* 

e. The training coordinator was a professional psychologist from the Corporate 
headquarters, who was experienced in organizational development and OD re- 
search methodology. 

f. The contents and methods can be outlined as follows: 

1) Several preliminary meetings and interviews were held between the develop- 
ment coordinator and key executives to develop strategy, determine what the 
pressing problems were, stimulate involvement and commitment, and provide 
an opportunity for the coordinator to understand the plant's communication 
channels, methods of decision-making and approaches to problem solving. 

2) The first half day of the development laboratory proper, was devoted to 
group discussions in which the objectives were more specifically defined and 
refined, and previously gathering information fed back to the participants. 

3) Then, the team was divided into four small groups for an exercise in com- 
munications, the purpose being to support the use of behavioral feedback and 
"process-observation." 
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4) Next, the team worked in groups on a priority listing of issues that they felt 
should be tackled during the rest of the lab. 

5) This was followed by a group decision-making exercise tc facilitate the 
team's ability co use the knowledge, experience and interests of the members. 

6) For the remainder of the lab, the team worked alternately in small groups 
and together, on goals, commitments and objectives for the next year. Inter- 
spersed were theory sessions and exercises (such as brainstorming) designed 
to facilitate team goal-setting. This resulted in ar outline of action 
steps for achieving the mutually -determined goals. 

Evaluation: 

a. The types of evaluation used were participant reactions, subsequent behavior 

changes as perceived by two top executives, and improved operating results. 

b. Methodology: 

1) Participant reactions were determined by a group interview at the close of 
the laboratory. 

2) Behavior change observations were obtained by means of interviews with the 
Division Manager and the Works Manager, the top executives in the or- 
ganization. 

3) The Works Manager reported on subsequent cost reduction figures and his 
assessment of the extent to which the plant was later meeting its per- 
formance goals. 

c. Results: 

J There was consensus among the participants that the group had met all the 
individual and group goals which had been set, particularly in improving 
communication and decision-making skills, and developing a set of group 
goals, with action plans, for the next year. 

2) The Division and Works Managers reported the following outcomes: 

a heightened sense of enthusiasm and team work, not just among the 
members of the management team, but also among their subordinates. 



- Less defensiveness and decreased concern about "personal stakes" and 
status. 

- Less "Yes- sirring." 

- Greater effort to clarify task assignments and objectives. 

- Increased volunteering of creative ideas for influencing sales. 

- More commitment and eagerness to make a concurrent reorganization 
work, and less widespread anxiety and covert resistance, than normally 
expected and v:^ countered. 

- However, it was felt that the group needed to give more emphasis to 
short-term goals and plans, as they were concentrating quite heavily 
on long-term planning. 

3) The Works Manager cited the fact that cost reduction figures improved by 
59 % two months later. He also felt that the lab experience was directly 
responsible for the fact that the plant was subsequently meeting all per- 
formance goals even though they were higher the previous year‘s, and 
volume of business was at an all-time low. 

4) An additional outcome of the lab was a shared perception in the plant that 
there is value in groups spending time in team building in order to most 
effectively use the resources of the group members. 

4. Conclusions: 

This form of organizational development, or "team building", has demonstrated 
value in helping management determine and achieve its objec /es. Furthermore, 
there was substantial evidence that the effects were extended beyond the participants 
to lower levels of the organization. 
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1. Lockheed Aircraft Corporation, Burbank, Ca. 

2. Lockheed Management Institute 

a. Objectives: 

1) To increase understanding of management policies, principles and practices. 

2) To improve effectiveness on present job performance. 

3) To accelerate preparation of participants for positions of higher re- 
sponsibility. 

b. Participants were several hundred managerial and staff personnel, ranging from 
supervisors to division managers, representing a wide range of functions through- 
out the Company. 

c. The setting was the campus of the University of Sothern California. 

d. The program is held several times each year, and lasts three weeks, full time. 

e. The trainers consist of Company personnel and USC faculty members. 

f. The curriculum consists of highly participative classes, using case-study methods 
for the most part. The contents include a wide variety of modern business con- 
cepts, as well as the philosophy and rationale of Company practices. A typical 
session covers the following topics and devotes approximately the following 



percentages to each topic: 

Managerial skills & problem-solving 25% 
Management control & operating problems 23% 
Interpersonal relations 23% 
Information about Lockheed 17% 
The external environment 6% 
Communications 6% 



3. Evaluation 

a. The effectiveness of the program was measured in terms of; reactions of the 
participants, their subordinates, other managers, top management, and USC faculty 
(to get at observed behavioral and attitude changes); and increased knowledge. 

b. The primary objectives of the research were to 1) determine the effects of the 
Institute on knowledge, skills and attitudes; and 2) to identify opportunities 
for improvement. 
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The research effort was designed to focus on the objectives, faculty, participants, 
content and structure of the Institute. Following are only the (cost salient methods 
and results of an extensive research effort. A representative sample of each of 
the above groups was interviewed to determine their reactions toward the Institute, 
how it could be improved, and what impact it had on the subsequent performance of 
the participants. 

To get at the effects of the Institute or the participants' knowledge, a test of 
the facts and principles presented at the Institute was administered before and 
after their attendance, 
c. Results: 

When the results of the various steps were combined, certain benefits of the 
Institution became especially apparent: 

1) The establishment and strengthening of inter-organizational comnunications , 
resulting from the heterogeneous mixture of the participants. Participants 
and observers frequently reported that inter-departmental functional relations 
were clarified, and relations with personnel in other parts of the organization 
were facilitated. 

2) Forty percent of the participants interviews stated specific occasions in 
which they actually used newly learned principles in relating to their 
subordinates. 

3) On the knowledge test, there was ? significant average gain after attendance 
at the Institute. (No details were reported in the source document.) 

In conclusion, analysis of the interview results, in relation to the training 
needs expressed by top management, showed that the Institute was particularly 
helpful in promoting improved communications. Increased understanding of other 
functions, more effective interpersonal relations, and more efficient problem-solving. 

However, the research findings also showed that to increase the effectiveness of the 
Institute, it would be useful to Improve the followl:«g characteristics of the 
participants: 
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a. motivation to improve one*s managerial skills 

b willingness to accept responsibility 

c. enthusiasm and energy 

d. receptivity to change. 

The researchers concluded that Increased attention should be given to establishment 
and communication of detailed Institute objectives. 
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Lockheed Aircraft Service, Ontario, Ca* 

Seminar ami Practlcum on Principles of Motivation. 

s. The objective was to develop the participants in "finding better ways to do things" 
In other words, to Increase productivity by involving employees more in solving 
problems related to their jobs. 

b. The participants were five first line group supervisors, representing the machine 

shop, metal fabrication. Inspection, planning and scheduling, and purchasing. 

* 

Participation was on a voluntary basis. 

c. The seminar was held on company premises. 

d. There were four meetings, once a week, of about l^ hours each. 

These training sessions were Interspersed with two or three meetings between 
the supervisors and their work groups. 

e. The trainers were two professionals from the Company's Industrial Relations and 
Personnel staffs. 

f. The training sessions consisted of group discussions on employee motivation, 
stressing workers' needs for feelings of achievement, responsibility, recognition 
and reward (Herzberg's "motivators" or "satisflers" )• Additionally, the 
participants received instruction in how to conduct meetings with employees, 
with emphases on listening and postponing evaluative responses to their ideas. 

After the first two sessions, several problem- solving meetings were scheduled, 
in which each participant met with his subordinate work group to apply his 
knowledge and practice the skills he had been exposed to during the first two 
sessiotth. Thus, the participants took on the role of trainers, under the 
observation of the Personnel and IR representatives. 

The first meeting in each case was focused on "finding better ways to do things", 
a sort of brainstorming session, during which the suggestions were listed on 
a large pad. The supervisors's role was to help clarify the problems, and 
later to lead discussions. In subsequent meetings, the groups discussed each 
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problem and possible solutions, in short, the supervisors, in several work-group 
meetings, interspersed with their own instructional sessions, practiced what they 
were learning in a real-life setting, on real problems. Thus, theory and 
practical application were alternated over a period of several weeks. (Note 
that higher levels of supervision were present at the last meeting.) 

Evaluation 

a. The effectiveness of this supervisory training program was measured in terms of 

1) changes in performance data; 2) changes in employee attitudes; 3) participant 
reactions. 

b. Methods: 

1) at the start of the project, measures of performance varied in the groups 
from very detailed and comprehensive to almost nonexistent. 

a) In the metal fabrication section (including the machine shop, for this 
purpose), productivity figures over time were measured in terms of 
•‘percent realization.** 

b) In the planning awl scheduling group, productivity was measured in terms 
of the release of shop orders. 

c) In the inspection group, productivity was measured in terms of the 
average number of hours of elapsed time between completion of production 
of items, and completion of their inspection. 

d) In the purchasing group, productivity was measured in terms of items 
per buyer man-hours. 

2) Employee attitudes were measured by means of a 23-itera questionnaire 

administered before and after the training program. The items related to 
employee participation and supervisory behavior related to employee 
motivation. (The sample size was not indicated in the source document.) 

*) Participant reactions were measured by means of taped interviews. 

» Results: 

1) a) In the metal fabrication section, productivity rose steadily over 
an 11-week period, from an average of 86 to 98. 
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b) In the planning and scheduling group, productivity varied greatly during 
the measurement period, but the modest average gain was considered in- 
conclusive. 

c) In the inspection group, the average inspection time declined from 5.5 to 
2.9 hours, from June to September. 

d) In the purchasing group, the cumulative average of items per buyer 
man-hour rose from 2.1 to 2.6, from July to November. 

2) On the attitude measures, the following average changes were observed: 

Metal fabrication -12% favorable 

Machine shop - 5% •• 

Planning & Scheduling (analysis not done) 

Inspection + 5% favorable 

Purchasing +18% ” 

In addition, detailed item analyses were made, which showed specific areas of 
attitude change. 



3) Participant comments were not quantified or categorized, but a thematic 
analysis showed that the participants felt that they had gained a better 
understanding of their own and the Company's problems; communications were 
improved; employees identified more with management; but in some areas, the 
workers felt that management did not view their suggestions in the cooperative 
spirit intended. 

Conclusions: 

The results were generally favorable. However, because a number of concomitant 

changes were made, e.g. , in supervision and work load, the results could not 

conclusively be attributable to the training program. However, it was concluded that; 

a. At the conclusion of the project, measures of productivity showed some improve- 
ment in every group. 

b. The program was not sufficient to significantly change attitudes. 

c. Verbatim comments of participants show a generally positive feeling. 

d. Perhaps the greatest value of the survey was to show that some serious problems 
still exist and appear to warrant management's attention. 



-30- 



32 



1. Union Oil Co,. , Los Angeles, Ca.(8) 

2. Seminars in Organizational Relations 

a. Objectives: 

1. "To expand the participant’s understanding Q f himself, including sinu lari ties 
and differences between his self-concept and how others perceive him. 

2. "To increase his understanding of the motivation and behavior of others. 

3. "To improve his understanding and use of the management process. 

4. "To increase his communication skills. 

5. "To improve his skill in exploring and analyzing problems involving people. 

6. "To improve his decision-making ability. 

7. "To increase his understanding and skill in interpersonal relations through 
using the Seminar as a clinical experience in which to experiment- -with 
minimum risks and penalties--in relating effectively to others. 

8. "To increase the knowledge of the participant concerning the operations and 
personnel of company functions outside his own department. 

b. 99 middle managers and technical specialists; about half were from the research 
and marketing divisions, the remainder from the whole spectrum of other divisions 
in the organization. They represented the second, third, & fourth echelons. 

Six seminars were held, from 1962 to 1966, each with about 20 participants. 

c. The seminars were conducted in the corporate headquarters facilities, & sponsored 
by the Management Development Department. 

d. Each seminar consisted of a 20-hour workshop in eight weekly or bi-weekly sessions, 
of about a half day each. 

e » The seminars and the research were led by the author of the referenced source 
document. 

f. The seminar was a highly participative educational experience, but not as 

theoretical as a diversity program. Content areas included motivation, inter- 
personal relations, communications, authority relationships, organizational 
change, planning, decision making, & m a n a gem ent policy. Methods included 
lecturettes, case studies, assigned articles* oral reports & role playing. 
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3* Evaluation 



a. Reactions of the participant to the program, and opinions of their colleagues 
formed the bases of the evaluation. 

b. Methods: 

The participants* reactions were determined by two means: 

1) discussional summaries, at the last meeting; each attendee orally identified 
those aspects of the seminar that felt were most and least useful. 

2) Each participant anonymously filled out a 3-page check- list questionnaire, 
covering the various aspects of the seminar. This was done at the completion 
of the seminar, and again three months later. 

The opinions of the participants* colleagues, concerning the impact of the 
seminar on participant's behavior, were obtained by informal interviews, 

c. Results: 

As the results from the two measures of participant reactions are parallel, the 
questionnaire results sum it up. (See conclusion below for summary.) More 
interesting were the results of the interviews with their colleagues, which 
were summarized as foil' -a: 

1, "An improvement in the attendee's communicating with his associates, 

2, "A more favorable attitude toward his work. 

3, "Increased interest in areas beyond the narrow confines of his immediate 
assignment. 

4, "Better relations with members of departments other than his own. 

5, "More interest (and skill) in helping to solve departmental problems, 

6, "Greater interest in his work. 

7, "More attention to collecting pertinent data prior to making a decision." 

(In some cases this was reported as a negative effect, in the sense of 
taking longer to make up one's mind.) 

There were scattered negative responses in the interviews as well, especially 
on the part of older associates; and these focused largely in the area of 
the extra workload imposed by the attendee's absence from his regular work 
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and the futility of off-the-job training of whatever kind. The negative comments 
occurred largely among peers rather than among supervisors or subordinates, which 
may have indicated the presence of some bias. This possibility is heightened by 
the clear recollection that no such comments originated with respondees who had 
themselves attended the Seminar, 

Conclusions (verbatim, from the unpublished source document, a roaster's degree thesis). 

1. In general the Seminars, in the opinion of the attendee, their associates and 

the Seminar leader, constituted a reasonably successful project in the development 
of managerial and professional personnel. They appear to have contributed to the 
personal growth and career progress of participants. 

2. The process used to select attendees suffered to some extent from the tendency of 
executives to use the Seminar occasionally to improve the morale of a restive 
employee regardless of his career potential. Perhaps in future projects of 
similar nature the impact of this tendency upon selection can be reduced. 

3. The use of participative instructional techniques for managerial and professional 
attendees was not only favorably received, but appeared to produce better results 
than were secured through non- participative methods. They are recommended by this 
experience for broader use in all developmental projects of related character. 

4. The Seminar experience , supported by the Survey findings that indicate similar 
courses in many other large corporations, would appear to endorse the benefit 
of this kind of in- company management development, 

5. Both the Seminar experience and Survey findings, supported by a great deal of the 
literature of personnel development and by the related experience of university 
graduate programs, appear to endorse the effectiveness of business case studies 
as an educational medium, especially for adult audiences of considerable exper- 
ience in the case areas. 

6. The use of role playing in the Seminar type of setting, while it gives evidence 
of being both beneficial and widely used, may still require further refinement 
in an effort to improve its acceptance by adult business audiences. This is an 
area that deserves continued exploration. 
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Including a broad cross-section of functional specialties and a reasonably diagonal, 
vertical echelon distribution in the Seminar group appears to have many values* 

It is a practice that should be extended in future projects of similar nature. 

The combination of managerial and professional employees (or functional specialists) 
in a Seminar group has values for both; it improves their understanding of each 
other and of each others' pi oblems; and it occasionally awakens unrealized 
interests, thereby providing career stimulus. 

The treatment of Business Ethics in courses similar to the subject Seminar is 
evidently not common; the Seminar experience, however, would recommend its 
broader coverage. 

Having attendees make oral reports on brief articles treating significant issues 
related to the management process proved to be an effective developmental medium. 

Its wider use is recommended. 

The conduct of projects like the Seminar provides the specialist in personnel 
development a valuable opportunity to evaluate the career potential of attendees, 
provided that he allows for the many differences between the Seminar environment 
and the work situation. 

The Seminar environment even on an in-company basis affords great opportunity 
to the attendee to experiment in a low-risk setting with ideas and methods it 
would be much more expensive to test on the Job. 



Some Brief Summaries 



Much of the research in this field of inquiry, not surprisingly, is conducted on an 
informal basis, without any intention of publication. As a result, it is difficult to dig 
out essential details and present them in convincing and enlightening manner, so that 
others can profit from the organizations' experience* To illustrate, consider the 
following vignettes, which are based on fragmentary information obtained mostly from 
structured personal interviews with survey respondents* 

California Blue Shield , which is headquartered in San Francisco, reports some informal 
studies on the Impact of a "Supervising by Objectives" program* This is a highly 
structured 30-hour program for the lower three levels of management, designed by an 
outside consultant. Major topics include setting objectives, solving problems, making 
improvements, getting cooperation, training, motivation, discipline, counseling, and 
control--in short, a wide range of supervisory functions. So far, about 300 participants 
have gone through the program, in groups of about 12 each. 

A sample of 42 participants was chosen for research purposes. Their supervisors 
filled out questionnaires which indicates that, from the supervisor's viewpoint, the 
program had an unusually favorable impact on 28 of the participants, who inanediately 
began i. plementing their newly- learned knowledge and skills. 

These 28 established, then achieved, or exceeded , measurable work-related objectives 
within 30 to 90 days after the course, using action plans that required the application 
of the techniques learned during the course. The sampling (14% of the 300) indicates 
that 67% of the supervisors measurably improved their own operations in 30 to 90 days; 

28% claimed improvements but could not support them with statistics; 5% had not been 

able to work on their improvement objectives. No data are available on the cost 

savings or reductions these results achieved although they are claimed to be "substantial". 

The Dow Chem ical Company has had considerable experience with unstructured laboratory 
training as a part of its management development effort, and has performed research 

to evaluate its effects. Here are some representative capsule summaries of their 
findings, (9) 
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Self-reported behavioral change information was obtained from 88 managers, two 
months after laboratory training, by means of a 19-item open-ended questionnaire. Their 
responses were then coded by the researchers into positive, neutral and negative cate- 
gories, There were significantly more positive than negative responses, but also a 
large number of neutral responses, indicating a need to refine the instrument. It was 
also found that those managers who had attended “off-site'* training; sessions showed 
significantly fewer negative responses than those who attended sessions on Company 
premises. This finding was interpreted to support the value of holding such training 
programs away from the organizational setting. 

Four independent studies, with samples of 24, 22, 12 and 14 managerial sensitivity 
trainees, were made, to test the hypothesis that trainees should be more accurate in their 
self-evaluations after training than they were before. Adjective check lists, varying 
in length from 12 to 22 items, with a 5-point rating scale on each item, were filled out 
by each trainee for himself, and also for each trainee by some of his colleagues (in some 
cases, by his fellow trainees, in others, by work associates). This was done both at the 
start and at the completion of the 40-hour laboratory session, and trends in the differences 
between self-ratings and peer-ratings were compared. 

The results in none of the four studies supported the hypothesis. More detailed 
analysis of one group, however, showed substantial Increases in the correlations betweei; 
self-ratings and peer-ratings from the ••before” to the ”after” condition, for 9 of the 
14 participants. As their peer-ratings were quite consistent, this was interpreted to 
mean that these 9 trainees seemed to have learned something about themselves during the 
training session, and hence they were able to see themselves more like others see them, 
after the training. 
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General Dynamics, Pomona Pi vis ion » Ca, 



This organization, which is widely recognized within the Aerospace Industry as a leader 
in human resources development, has an interesting approach to validating its management 
development efforts. Instead of research on the effects of a specific program, (the 
company sponsors a wide variety), they consider the collective impact of the programs 
on the company's human resources, in the*fol lowing manner. 

As a part of a company-wide manpower planning and administration program, the Division 
places a high degree of emphasis on manning tables and backup charts for all management 
positions as a guide in determining management growth, objectives, both on an individual 
and collective basis. A few years ago, a study showed that cnly 67% of the management 
slots had ready backups, that is, collateral or subordinate candidates who were prepared 
to take over the position with only minimal additional development. (This assessment is 
based on the pooled opinions of the incumbent of the given position, his superior, and 
a representative of the Personnel Department.) Over a three year period, this percentage 
of ready backups was raised from 67% to 90%, through a variety of development programs, 
including on-the-job training, professional seminars, and university short courses on 
company time, extensive in-plant courses covering both technical, administrative, and 
management fields, after-hours classes sponsored by the Management Club, and a wide 
variety of tuition refund educational programs dealing with a great cross-section of 
colleges und universities in the Southern California area. Highly specialized courses 
for selected personnel are identified for advanced graduate education at the MS and PhD 
levels generally under full-time educational programs. In short, in the view of line 
management, their management development program's effectiveness was gauged in terms of 
an overall strengthening of the pool of management talent. The company uses a compre- 
hensive, systematic approach to human resources development, carried on down to the 
individual level; it is a coordinated effort between the employee, his manager, and Educa- 
tional Development Counselors. To the maximum extent possible, this joint effort is 
implemented and monitored by educational/management specialists. 
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In addition to the above, the Division sponsors very strong in-plant activities associated 
with Career Development and Management Intern Programs whereby highly selected personnel 
identified as high potential are given extensive and refined training closely allied with 
in-plant functional upgrading. 

In support of their overall upgrade programs, maximum effort has been placed upon 
establishing an in-plant development program for first-level supervisor /managers to 
meet the growing demands for excellence in this sector of management. 

Professional level university courses are developed throu^ joint effort of company 
training representatives and university department chairmen. 

In addition to this gross measure of effectiveness, the Division has used reactions of 
trainees, and experimental written information tests, on a before-after basis, to get 
at learning. 
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Lockheed Aircraft, Burbank, Ca c 



Like several other sophisticated companies covered by this survey, Lockheed has become 
involved with a form of organization development called "team building", which seems to 
te paying off on the ledger sheet. Here are some excerpts from an unpublished report, 
that pretty well sums up two applications: 

"Lockheed, in its approach to team** building has bypassed the lengthy, costly, 
and job-removed preparatory aspects of organization development s^h as the 
Managerial Grid aid sensitivity training. The groups consist of work teams 
and the sessions are work oriented; however, with the consultant* s help, the 
group cakes time to view the "process" of the meeting, to examine the effectiveness 
and freeness of communications. It is felt that this combination, compressing 
the experience, has tipped achieve the good acceptance by busy executives." 

2. "Work groups were called together for meetings in which impediments to production 
were examined and corrective actions sought. The psychologists insured that there 
was follow up on e \ch item. As appropriate, ••multi-level" meetings were held in 
which workers representing the work group fed their suggestions to higher manage- 
ment levels who were present, and these were explored on a problem solving basis. 
These meetings were almost entirely work-related, as opposed to concentrating on 
working conditions, employee policies, etc. Preliminary results showed that 

cost savings resulting from the meetings were equal to 12 times the cost of the 
program." 

Ro berc B. Morton and Associates of Sacramento report two promising studies on the impact 
of OD programs designed to increase personal and organizational effectiveness by breaking 
down barriers between the various management groups. 

1. In one anonymous organization, they evaluated an OD program conducted by company 
personnel. Three hundred and eighty managers from several levels of management 
of one division attended OD for eight days; a comparable division was used as a 
control group. After OD, Morton found that the experimental group was superior to 
the control on: profitability; more confronting of organization problems and risk 

taking; less conformity; supervisory skill (via Mahler's Coaching Practices Survey, 
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completed by the trainees' subordinates); higher Internal Control on Morton's 
adaptation of Rotter's Internal-External Control (I-E) Scale(lO); less discrepancy 
in I-E scores between managerial levels. 

2. This last finding was replicated in another anonymous organization, employing Morton's 

Organization Development Laboratory. Using the 1-E scale in a before and after design, 

with voluntary anonymous participation, he found significant increases in total 

Internal Control scores and significant changes in the expected direction on four 

of the five subscales: Inevitability; Futility; "Can Do'*; and Earned Effort; but not 

on Change. These results must be interpreted cautiously, in view of the discrepant 

sample sizes for the pre- and post-tests, which were 59 and 73, respectively. 

However, they seem to indicate an increased sense of personal responsibility, and 

* 

perhaps increased achievement motivation, presumably as a result of the QD Experience. 
Practical Management Associates, Inc. , Woodland Hills, Ca . 

This consulting organization has an interesting built-in mechanism to continually validate 
the effectiveness of their 3-day middle management development seminars. These seminars 
are highly participative, use primarily a case-analysis approach, emphasizing problem- 
solving in "real-life" organizational management. 

Each participant comes to the program with a problem that he and his superior have 
identified and agreed on. At the end of the 3-day seminar, he is expected to come up 
with a plan to attack that problem. Then, some time later, after returning to the job, 
he reports back to the consulting firm on the disposition of the problem. This approach 
has been so successful in demonstrating to the client companies that their investment 
pays off in improved organizational performance, that PMA has an unusually high 
rate of repeat business with their client companies. In fact, they reported an 
increasing volume of business even during the 1969-1970 recession, when most other 
management training consultants were experiencing a serious decline in volume. 

However, PMA has not compiled their feedback information into the kind of formal summary 
report that would be useful for scientific purposes, because PMA believes that the 
standard for evaluation varies from one organization to another and that the same standard 
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of effectiveness cannot and should not be applied to all organization. Hence their 
measures of effectiveness may be of limited interest for a re search- oriented survey 
of this sort. 

Wells Fargo Bank , of San Francisco, reports the results of a combined O.D. and job 
enrichment program in one central department. While this program resembles traditional 
management development only slightly, it will illustrate an alternative approach that 
is gaining some currency in development circles. 

Wells Fargo’s approach is to involve all employees in a program of planned change, 
to improve productivity and morale through the application of Job enrichment principles 
advocated by Herzberg. The program and research were administered by four members of the 
corporate management development staff. 

The subject department was divided into a test group and two control groups, of about 
15 members each. Work changes were introduced (in the test group only) and carefully 
monitored over a six-month period, to minimise the ir Hawthorne Effect." 

Follow-up evaluations revealed a marked decrease in overtime in all three groups, 
particularly in the test group. The number of customer- reported errors per 10,000 items 
processed, also declined In all three groups. (Improvement in the test group was not 
greater than in the controls, in part because supervisory inspection of work was 
decreased from 100% to 20%, as a part of the job enrichment, in which employees were 
made responsible for checking and correcting their own errors.) Absenteeism and 
voluntary turnover were also subsequently less in the test group. Finally, questionnaire 
results showed Improved job satisfaction in the test group. (The questionnaire was not 
administered to the control groups.) 

It was concluded that "although the job enrichment/organiaation development results 
are encouraging, ... this is not a panacea for all the Bank's problems, nor will it 
necessarily work in all cases. ...Job enrichment must be carefully introduced Into any 
department and requires a careful preassessment of the conditions of the department and 
the value system of the managers involved." 
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STUDIES USING ONLY INTERNAL CRITERIA 






1. North American Rockwell, Power Systems Divisions, Canoga Park, Ca. 

2. Understanding the Why of Affirmative Action. 

a. The objective was to increase self-awareness and individual sensitivity in the area 
of minority group problems. 

b. The participants were 19 middle managers, who were identified as being potentially 
able to affect a large number of employees. Several black managers and one female 
supervisor were included. 

c. The sessions were held at company training facilities in Canoga Park. 

d. There were seven 3— hour meetings, held once a week. 

e. The instructors were two black professionals, one an administrator from San Fernan- 
do Valley State College, the other from a private agency in the Los Angeles area. 

f. The sessions consisted mainly of assigned readings and group discussions on black 
history and civil rights. 

3. Evaluation : 

a. The effectiveness of the Seminar was gauged in two ways: 1 ) reactions of the par- 

ticipants; 2) changes in scorer, on a written quiz desiyn^d to measure "awareness". 

b. Methods 

1) The reactions of the participants were determined in a group critique at the end 
of the last meeting. Also, written, free-form comments were requested, and sub- 
mitted after the conclusion of the program. 

2) The written quiz, consisting of objective multiple-choice and true— false ques- 
tions, was administered before and after the program. Maximum possible aware- 
ness score was 34. 

c. Results 

1) In the group critique, a substantial majority of the participants expressed the 
belief that the Seminar was workable, and that it should be continued. Some 
emphasized the importance of top management being exposed to the Seminar, others 
felt that lower levels of supervision would profit the roosi. One participant 
expressed doubt as to the Seminar's value, while four did not conrxt themselves. 
The written comments were generally favorable and most recommended the inclusion 
of top management in subsequent seminars 
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2) On the awareness test, the pre-test mean score was 16.2 and the post-test mean 
score was 22.1. The largest increase was 23, and 3 participants showed no 
increase. (The statistical significance of the changes was not indicated.) 
Conclusions 

By all the measures applied, a significant amount of learning took place in the pilot 

Affirmative Action Seminar. The experience in the eyes of most participants was both 

interesting and worthwhile. Most recommend continuation of follow-on seminars for 

other supervisors. Benefits to the company may be expected in the form of more 

positive resolution of human problems and a predicted improvement in the acceptance 

« 

and attainment of affirmative action goals. 



- 46 - 




PEDR Urban Associates, Inglewood, Ca. (11) 

Urban Executive Leadership Seminars for Black Professionals 

a. Objectives: "to assist black professionals in achieving a me e positive and inte- 

grated personal identity, in further developing their organizational skills, and in 
fashioning appropriate urban executive roles in the community." 

b. Fifty one black professionals, from a wide variety of organizations in the 

Ixjs Angeles area, divided into two groups of 20 & 31 participants. Qualifications 
for participation included: 25 to 35 years of age; at least a bachelor's degree; 

at least two years of supervisory or management experience. 

c. Seventeen seminar sessions were held at various locations in the Los Angeles area, 
including universities, government facilities and private organizations. 

d. Three weeks, concentrated. 

e. TVo professional team trainers and several guest speakers & discussion leaders. 

f. "Several new approaches in both curriculum & teaching methodology were used. They 

included: (1) an emphasis on personal, professional & social values; (2) an inte- 

grated behavioral science curriculum covering self-identity and individual behavior, 
interpersonal relations, group dynamics, organization and management, intergroup 

relations, and planned community development ; and (3) a teaching approach 

using individual and group teaching methods rooted in both the attitude change 

theory of social psychology and the self-awareness techniques of humanistic 
psychology." (12) 

g. A large variety of teaching methods and contents, too numerous and complex to 
present here in detail, were utilized. Included were an orientation session to 
establish objectives, a two-day retreat, group analyses of assigned readings and 

movies, role playing, modified sensitivity sessions, field experiences, and guest 
lectures. 

Evaluation 

a. Learning (in terms of measured attitude changes), and self-reported behavioral 
changes and expressed reactions, were used to gauge the effectiveness of the 
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program. 



b. Methods: 



A research design was used which measured the executives' attitudes before the 
program began, again at its conclusion, and finally several months later. In 
addition, the attitudes of a control group, which did not take part in the seminar, 
were measured before and after the seminar. 

The major hypothesis of this study was that participating executives will experience 
significant attitude changes toward a more positive view of self, view of othe % 



and view of the environment. 

Selected scales from seven valid and reliable test instruments were used as oper- 
ational measures to evaluate changes in various dimensions of three attitude 



clusters; view of self, view of others and 
1) View of Self 
Dimensions 

* Self Concept 

*** Independence 

* Achievement motivation 

* Leader Identity 



view of the environment. 



Measures 

Shostrom's Personal Orientation 
Inventory (POD 

POI; Gordon's Survey of Inter-personal 
values; and Schutz's Fundamental Inter- 
personal Relation Orientation (FIRO-B) 

SherwoOd ' s vAch scale. 

Gordon's Survey of Inter-personal 
Values 



2) View of Others 
Dimensions 

** Human Nature 
* Interpersonal Attitudes 
*** Philosophy of leadership 

3) View of Environment 
Dimensions 

Emotional flexibility 
Mental flexibility 
Value flexibility 



Measures 

POI 

Gorton's Personal Inventory and FIRO-B 
Haire's Leadership Questionnaire 

Measures 

POI 

Rokeach's Dogmatism Scale 
Gordon's Personal Inventory 
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c. Results 



Asterisks in front of the above dimensions indicate statistically significant 
changes, as follows: 

* - significant change, in the hypothesized (favorable) direction, in Seminar 
Group I. 

** = ditto, in Group II 
*** = ditto, in both Groups I & II 
d. Comments 

The above results were followed up two months later. TWelve participants from 
Group I retook selected attitude scales, which indicated that the immediate changes 
persisted, and in most cases improved further. 

The effectiveness of the program was also measured by expressed reactions of the 
participants in both groups. There was general agreement that the seminar objec- 
tives were achieved. A majority stated that they had improved in their ability to 
plan and use time more effectively. M'st also agreed that they had learned tech- 
niques that allowed them to more effectively evaluate their own personal values and 
professional development. 

Conclusions .. These results justify the conclusion that executives who took part in this 
seminar experienced att.uude change. Analysis of this data shows that changes in 
Seminar Group II were of less magnitude than those in Seminar Group I. This result is 
probably a consequence of having fifty per cent more participants in Group II and 
shorter Seminar sessions made necessary to accommodate their professional work 
schedules. The most significant attitude change occurred in executives' view of 
themselves and in their view of others. Members of both seminars strengthened their 
se If -concept and became more Independent. Executives in Seminar Group I increased 
their desire to achieve success and also their desire for leadership. Executives in 
both seminars adopted a more democratic philosophy of leadership while those in Semi- 
nar I reduced their desire to control others, and those in Seminar II developed a more 
positive view of man. Only in their “view of environment" did participants fail to 
make attitude changes during the seminar. 
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1. PEDR Urban Associates, Inglewood, Ca. (13) 

2. Urban Executive Leadership Seminar for Mexican- Ameri can Professionals 

a. Objectives: 

(1) "assisting participants to achieve a more positive, integrated" self concept 
by encouraging them to re-examine and synthesize personal, professional and 
community values; 

(2) "sharing current applications of behavioral science research findings and 
assisting participants to develop related skills in individual behavior, 
interpersonal relations, group dynamics, organization and management, 
intergroup relations and planned community development; 

(3) "providing participants an opportunity to build working relationships with 
their peers and with leading business, professional and political leaders in 
both the Mexican-Araerican and Anglo communities; 

(4) "guiding participants in the development of a positive role in the community, 
by planning a community development project that would provide experience in 
community problem solving; and 

(5) "encouraging participants to explore their personal identities by studying the 
contribution of the Mexican-Araerican to the development of thn Southwest." 

b. 25 Mexican-American executives from a wide variety of organizations in the Los 
Angeles area. Selection was based on the individual's management experience and 
on his potential for making a significant contributioi to ’•he community. 

c&d. TWenty seminar sessions spread out over a seven-week period in 1969. 

e. Seminar leaders included professional trainers from PEDR, represer tatives from the 
Mexican-American Industrial Opportunity Council and from the Institute of Executive 
Training, and numerous guest speakers. 

f&g. The contents, methods & formats were similar to those used in the seminars for 
Black professionals -- see p. 43. 

3. Evaluation 

(Note: the following summary, derived from the source document, is partly verbatim and 

partly paraphrased.) 
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The type of evaluation may be classified as a measure of learning, in terms of 
attitude change, as contrasted with information retention. 

A battery of attitude tests was administered in the first and last sessions of the 
program to evaluate changes in various dimensions of three attitude clusters: 
view of self, view of others* and view of the environment. The variables and 
instruments used were as follows: 



(1) View of Seif 




Dimensions 


Measures 


* Self Concept 


Shostrora' s Personal Orientation Inventory 
(POI) (two scales) 


** Seif Direction 


POI; Gordon's Survey of Interpersonal 
Values; and Schutz's FIRO-B 


* Achievement Motivation 


Sherwood's vAch scale 


Leadership 
(2) View of Others 


Gordon' s SIV 


Dimensions 


Measures 


* Human Nature 


POI - Nc Scale 


Attitudes toward others 


Gordon's Personal Inventory & FIRG-B 


Philosophy of leadership 
(3) View of the Environment 


Haire's Leadership Questionnaire 


Dimensions 

Emotional flexibility 


Measures 


*** Mental flexibility 
Value flexibility 
Results 


= POI (six scales) 

Gordon's Personal Inventory 
(one scale) 



Asterisks in front of the above dimensions indicate statistically significant 
changes, in the hypothesized (favorable) direction, as follows: 

* = significant change on one scale 
** = significant change on two or three scales 
*** = significant change on 3 of the 7 scales 
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d. Comments 



Unlike the report on the seminars for Black professionals, participant reactions 
and delayed follow-up attitude measures were not reported for this seminar. 

4. Cone lusion . l, The before-after results of six attitude instruments encompassing 
twenty-five separate scales show that the Hex lean- American professionals who 
participated in the Executive Leadership Seminar experienced significant attitude 
changes as a result of their seven-week experience together. These changes mainly 
occurred on two dimensions - attitudes related to their view of themselves and to the! 
image by increasing in self regard and becoming more self -directing. At the same time 
they developed more openness, flexibility and sensitivity toward the environment. 
Assuming a relationship between attitudes and behavior, it is likely that as a 
result of the seminar, these Mexican-American professionals will perform more 
effectively as executives and as community leaders." 



O 
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System Development Corporation: Santa Monica, Ca. (14) 

Management System Training Using Leviathan 

a. Objectives: 

1) "To provide an environment for the students to appreciate the dimensions and 
limitations of a complex managerial task. 

2) To develop a capability for effective management of what can be managed and 
the ability to "live with" the balance. 

3) To provide a setting for group interaction. 

4) To assess and develop abilities and skills. Including: 

. Identification of values, establishment of objectives, and criteria for 
measuring managerial performance. 

. Preparation of plans, commitment of oneself to them, and carrying them to 
completion. 

. Personnel assessment, assignment, and training. 

* Securing of the optimum performance from subordinates. 

. Understanding of informal organization and communication in large organiza- 
tions. 

. Management and utilization of information. 

5) To establish attitudes through "internalization" oy doing rather than ‘identi- 
fication* by intellectual activity." 

b. Participants: The study was done in conjunction with a regular graduate course in 

Organizational Behavior, at USC. "The class participating in the leviathan 
interaction consisted of 28 Air Force officers enrolled in a Master's degree 
program in R & 0 System Management (referred to as the RDSM program) and the 

USC Graduate School of Business. Ail class members had degrees in science or 
engineering, approximately one third of them from one of the military academies. 
Rank ranged from lieutenant through major; length of service from 2 to 15 years. 
Median age was 33 years." 

c. The setting was at System Development Corporation facilities. 



••The Complex Computerized Organization Simulation Program (CCOSP) was from the 6th 
through the 12th week of the course, with two sessions per week, each of 3 to 4 
hours* duration. The first three periods were for training in operation of the 
equipment. The fourth session was organizational, with the group choosing a 
leader, assigning other positions, and setting general policy. The remaining 
eight periods were spent operating the simulation. About every two hours during 
the last five sessions, play would cease while the instructor "debriefed" the 
group and led its members in discussing the causes and effects of their behavior." 
The trainers (or more appropriately, educational coordinators), were two professions 
psychologists with three assistants* 

"Leviathan is a computerized simulation for studying communication in large social 
organizations. It comprises a theoretical frame-work and a system of computer 
programs and employs a series of experimental simulations in a laboratory, 
experimental controls, and quantitative measures of organization performance. 

The method permits the conduct of experiments on the development of social 
organizations and the impact on individuals of participation in a feedback- oriented 
organizational environment. Leviathan programs simulate elements that arc common 
to a variety of organizations; the model operates within an environment created 
to provide an operational context for the interaction between the organization 
and the model. 

"Leviathan has fi *e basic features that can be useful in developing managers, 
•abstract* realism, complexity, human interaction, cooperation, and information 

handling." 

Several observers were present throughout the exercise. They were able to observe 
the participants via special mirrors without interfering with the proceedings. 

Note that the present summary is a drastic oversimplification of a complex situa- 
tion, which is explained in considerable detail in che source document. Parts of 
this summary are direct quotes; other parts are reworded and reorganized. 




Evaluation 



a. The effectiveness of the exercise was measured several ways, as detailed in section 

b. below. These can be categorised as: 

1) participant reactions 

2) attitudes toward various aspects of the exercise 

3/ changes in their managerial values (=learning?) 

4) feedback from observers 

5) judgments from achievement of objectives 

b. Methods: 

1) Participant reactions were obtained by means of: 

a) small group debriefings and class discussions interspersed throughout the 
exercise; and 

b) anonymous, written individual evaluations at the end of the course and 
again a year later. 

2) Attitudes were measured by a special 15-item, Likert-typ* questionnaire. 

3) Changes in managerial values were measured by means of a standard instrument 
called "A Comparison of Managerial Values, H which is based on Blake & Mouton's 
Managerial Grid. This test was given before and after the exercise. 

4) Feedback was given by means of free-form, written comments from three observers. 

5) Judgments about the extent to which the objectives of the exercise were met, 
were made by the instructors, 

c. Results (in terms of the above methods): 

l) a) “The responses to the broad questions generally indicated that the 
experience had been valuable; in fact the responses were extremely 
enthusiastic." (Several pages of comments are presented.) 

b) On the anonymous written evaluation, submitted at the end of the course, 
there were 20 favorable, 6 neutral and and 2 negative responses. On the 
follow-up evaluation a year later (covering the entire MBA program), 
several favorable comments indicated increasing appreciation for the value 
of the simulation experience. 
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2) On the attitude questionnaire, the results were as follows: The class was 

divided into upper and Lower groups on management responsibility in the Levia- 
than hierarchy, and the two groups' responses averaged separately and together. 
The results indicated that both subgroups felt that the experience was worth- 
while, and that the two subgroups did not differ significantly. However, the 
less structured feedback indicated that the class felt that the upper management 
had learned most. (This was confirmed by the next instrument.) 

3) On the "Comparison of Managerial Values" test, the top management group showed 

shifts away from 9,1 and 5,5 styles, and toward 1,1 and 9.9 styles (not signifi- 
cant, however). The lower management group shifted in the direction of l,l 
(significant). When the two subgroups were combined, they showed an overall 
shift toward 1,1 (significant). (This anomalous, unpredicted change to a theor- 
etically inferior style of management, on the part of the lower- level managers, 
was perhaps attributed to their heavy involvement with technological management, 
which did not give them an opportunity to interact with other people). According 
to one of the researchers, in a private conversation with the present reviewer, 
this interesting training outcome may have important implications for team 
training: beware of the possible mal-adaptive effects of relegating trainees 

to chronic subservient positions; rather, be sure to rotate trainees, so that 
everyone can practice the kind of behavior appropriate to genuine leadership. 

4) Feedback from the three observers covers about seven pages in the source docu- 
ment. Here are a few selected but representative comments: 

"The leviathan simulation represents a powerful program for management training 
and organizational design." 

"leviathan was universally perceived as much more valuable than a T-group training 
situation." 

"The Leviathan vehicle can be a valuable resource in training people to operate 
effectively in organizations, provided that considerable additional effort is 
devoted to the design of each experience, as contrasted to the design of the 



vehicle itself." 
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objectives, the instructors 



5) In evaluating the exercise in terms of the stated 

feel that "in summary, many of the objectives of the exercise were met; the 

remainder would probably have been met if the exercise had been two or three 
times as long. 1 ’ 

Conclusions 

"leviathan, or a simulation like it, can be a very valuable tool in management education. 
’•Whether Leviathan is the appropriate vehicle for this simulation depends upon the spec- 
ific purposes of the management education or training, and the cost/effectiveness of 
Leviathan in relation to other simulations." (A list of 10 desirable characteristics 

of a computerized simulation is presented, based on the investigators' experience 
in this program.) 
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1. University of California, Los Angeles, Ca. (15) 

2. Sensitivity Training Program 

a. The objective of the program is, briefly, to develop social awareness and percep- 
tual sensitivity in the participants, in order to promote their behavioral 
effectiveness, and personal and professional development. 

b. The participants were 67 middle and upper managers and professionals from a variety 
of organizations in the Los Angeles area. Most were men, ranging in age from 

mid 20* s to late 50»s, with about 75 percent between 30 and 50, They worked in 
5 T- groups of about 14 or 15 members each, 

c. The setting was at the UCIA Graduate School of Business Administration, except for 
an introductory weekend at the University's Lake Arrowhead conference center, 

d. The program lasted for five weeks, meeting weekly for about 4^ hours, including 
dinner, 

e. The trainers were five experienced professional T-group leaders on the UCLA staff, 
one for each of the five groups, 

f. The contents and format of the T-groups were typical unstructured, highly 
participative group discussions, in which the participants interacted in an 
environment conducive to trying out and receiving immediate feedback on a variety 
of interpersonal behaviors in a low-risk atmosphere. As there is no pre-deter- 
rained agenda and the actual events that occurred are considered confidential, it 
is not possible to elaborate. There is no indication that any unusual events 
took place, aside from the normal variations that characterize typical T-groups. 

g. Note that in the source document, the effects of T-groups were compared with the 
effects of two oiher forms of human relations training. However, as the subjects 
of the latter two groups were mostly students and housewives, they will be excluded 
from this summary, except for brief mention in section 3c. 

3. Evaluation 

a. The effectiveness of the program was measured in terms of learning, that is, 
attitude change; in addition, behavioral change was measured by means of a 
"seif scale" and questionnaires filled out by the participants' spouses. 
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b. Methods: 



The 67 subjects were split into two groups of 37 and 30. One group was administered 
the Gordon Personal Profile, the other the Gordon Personal Inventory, on a 
before-and-after training basis. Because of administration problems, there was 
considerable shrinkage in the post- test sample size, so that the following N's 
resulted: 

Number of Usable Tests 
Before After 

Gordon Personal Profile 37 34 

ft * 

Gordon Personal Inventory 30 20 

(Each of these two tests has four sub-scales.) 

To supplement the objective tests, on the post-test, each participant filled out 
a parallel "self-scale" that was designed to measure felt changes in the four 
dimensions represented by the test. Likewise, the participants' spouses were 
asked to fill out a similar questionnaire, indicating perceived changes, if any, 
on the same four dimensions. Altogether, 33 spouses participated. Finally, a 
sociometric instrument was filled out by the participants to get at factors of 
interpersonal acceptance and rejection, mutuality of ratings, and accuracy of 
perceiving how others will rate oneself, 
c. Results: 

Numerous hypotheses were tested, relevant to expected changes in the measured 
variables. On the objective tests, only one dimension. Responsibility, changed 
to an extent greater than could reasonably be attributable to change. The 
average score increased from 22.6 to 24.8; t* 1.75, p. 05. 

The results of the "spouse scales" were expressed in relation to similar measures 
made in the other two forms of human relations training included in the source 
document study. The T-group subjects were seen by their spouses to have increased 
more in Ascendancy than the Human Relations class members did; but less in Vigor 
than the Developing Personal Potential class members did. 

On the sociometric instrument, there were no significant results for the T-groups. 
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4. Conclusions 



While it is difficult to draw firm conclusions from one study, this research showed 
that it is difficult to gauge the effectiveness of sensitivity training in terms of 
scores on personality tests and questionnaires. There is suggestive evidence that 
participants may increase somewhat in such behavioral traits as Responsibility and 
Ascendancy, but these effects should be confirmed by cross validation studies before 
they are accepted with a high degree of confidence. Also, a larger battery of 
instruments might be more fruitful. 



Conclusions 



If the above research results are representative of current practices, perhaps a few common 
themes or trends in the field of management development, and research on its effectiveness, 
can be identified. However, since comparable reliable information from previous eras is not 
available, it is difficult to determine whether these are genuine trends , or merely a 
heterogeneous collection of snapshots of the present "state of the art" that have little 
in common with each other or with earlier efforts. . r :ore demonstrating more chaos than 

orderly, directional tendencies. Therefore, at the risk of overgeneralizing the findings 
and opinions of a limited and biased sample of respondents, the following conclusions are 
suggested: 

1) Management development programs take on a bewildering variety of forms, with an 
almost infinite number of philosophies and approaches. For survey purposes, 
"management development"* was very liberally defined, with the result that some of 
the programs reported herein might not even be recognized by some readers as 
management development at all; similarly, some readers may not recognize some of 
the offerings as "research". 

2) Many management development programs are based on a set of stated or Implied 
objectives, but the objectives are rarely operationally defined. 

3) In a large number of companies, "management development", with i£s traditional 
emphasis on personal growth, is being superseded or at least supplemented, by 
"organization development", which is more concerned with the environment or culture 
of the organization, locating and solving organizational problems (such as 
communications, planning, organizing, coordinating efforts, setting objectives, 
etc.), and making the kinds of changes that will in turn facilitate individual 

and team growth. 

4) Management (and Organization) Development programs are tending to be more closely 
integrated with other personnel programs, such as manpower planning* management 
assessment, performance appraisal, and human resources accounting systems. In 
short, development programs are being treated more like sub-systems within a 
larger framework of related programs, than as separate, semi -autonomous efforts. 
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5) Increasingly, organizations are relying on outside resources for putting on develop- 
ment programs, phasing out of large in-house training staffs, and retaining only a 
small nucleus of professional training administrators, who work with line management 
to design programs tailored more directly to their needs, (How much of this appar- 
ent "trend" is hard to determine). There seems to be less reliance on standard, 
classroom-like seminars through which supervisors and managers are passed on a 
routine assembly-line basis, and more emphasis on tailor-made programs for specific 
groups or individuals. 

6) The behavioral sciences are becoming increasingly influential in the theory and 
practice of development, with increased emphasis on motivation, problem-solving and 
decision-making, the development of cognitive and communications skills, and attempts 
to change value systems and attitudes; and less emphasis on personality and "human 
relations" matters, in isolation from work itself. 

7) Relatively few management development programs have been "validated" in the sense 
of being supported oy empirical evidence on their effectiveness. In fact, because 
of the difficulty of performing research in this field, it is likely that propor- 
tionally less research is being done on development than on management testing and 
assessment programs. 

8) The volume of research on the effectiveness of development programs is increasing 
rapid ly-- several doctoral dissertations, not reported here, have been found to be 
in process. Aside from a few training specialists who dismiss the very feasibility 
of such research (and the many who are not research oriented or are actively ooposed 
to it), there seems to be increasing awareness, on the part of organizational 
leaders and professionals, of the economic and social benefits accruing to a 
scientific approach to manpower problems in general. 

9) Researchers are becoming more sophisticated and creative in their research designs. 

As is the case with educational research, the question can no longer be simply 
stated in terms of "the effects of management development", but in terms of "what 
kinds of effects, of what kinds of experiences, for what kinds of subjects." 
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For example, a "rational" approach may have very positive effects on highly educa- 
ted managers, but little impact, or even maladaptive effects, on uneducated manag- 
ers; however, few studies have been able to utilize moderator variables or multi- 
variate research designs, because of small sample sized and inadequate resources. 
Our inability to exploit existing technology (such as the computer) and research 
methodology (such as advanced statistical techniques and politically unpopular 
psychological research instruments) too often results In . inconclusive and equivocal 
research "findings" that are practically useless for decision-making purposes* 

10) The presence of uncontrolled variables, such as organization changes and seasonal 
and secular economic trends that occur during the measurement period, makes it 
difficult to conclusively attribute criterion data to the management development 
experience. 

11) Even though the majority of the hitherto unpublished studies reported herein seem 
to support the effectiveness of management development programs, we should be 
cautious about accepting the conclusions at face value. This is because there is 
an unknown, probably high degree of bias in reporting only those studies that 
"work out." (This caveat, by the way, also applies to the published literature! ) 
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Recommendations 



The following recommendations are offered to organizations interested in evaluating the 

effectiveness of their management development programs. 

1) Management development program objectives should be specified in operational or be- 
havioral terms. That is, the criteria of performance should be so defined that a Judg- 
ment can readily be made as to the extent to which the program objectives have been met. 

2) If research on the effectiveness of the program is considered desirable, provision for 
the research should be designed into the program froa~ the beginning, not applied on a 
post-hoc basis. Special attention should be given to replication studies, because even 
with statistically significant results the effects may be unstable, or situatlonally 
anchored. 

3) Depending on the estimated value of such research in relation to its cost, the type of 
evaluation chosen should be the one with the most meaning for the particular program; 
ideally, in most cases, this would be some objective measure of results, such as 
productivity, efficiency, cost reduction, improved manpower statistics, etc. However, 
in the absence of such high level criteria of effectiveness, a lower level, such as 
trainee learning or attitude change, might be preferable lo not measures at all. 
Participant reactions are of severely limited value for scientific purposes, no matter 
how valuable they may be for other purposes. 

4) Whatever evaluation measures are selected, they should be tied into the stated objec- 
tives as specifically and explicitly as possible. 

5) Evaluative research should allow for the operation of moderator variables, (such as 
participants' value systems, attitudes, cognitive factors) that are likely to influence 
the relationship between the predictor (the development experience) and the criterion 
of effectiveness. 

6) Increased emphasis should be given to research on criteria of effectiveness, as an 
approach to validating not only development programs, but also the merit system as a 
whole. 

7) Increased emphasis should be given to integrating T & D programs with other manpower 
programs, particularly assessment and performance appraisal systems. This integration 
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effort might take the form of interdepartmental appointments, liaison officers, temp- 
orary interdisciplinary task forces, etc. 
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APPENDIX I 



Please return this form by June 30, 1S70. in the self-addressed envelope provided. 
Name 

Title 

Organization 

Address 



Telephone 



1 . 



ERJC 

imiflaffffifiiaaa 



e«eoMv. been l " vo,ved> durl "8 th « past few years, in research on the validity or 
effectiveness of management training and development programs? * 

If Yes, please indicate: 



Yes 



No 



A. The research included: 

I* Top level managers & 
executives 

2. Middle managers 

3. First— line managers: 

a. White collar 

b. Blue collar 



2. Do you know of any such research 
A. In progress? Yes No 



B. 



The research results were: 

1. Published 

2. Not published 

(If published, please provide references 
on the reverse, or attach separate 
sheets. ) 



C. The most convenient dates & w .imes: 
to telephone or visit 

you (during July only), would be 

1. (1st choice): 

2. (2nd choice): 



B. 



Completed recently, but not published? 
Yes No 

Yes, please give names & address on 
reverse, or attach separate sheets.) 



3. Do you wish to receive a copy of the final report of this study? Yes 



No 
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APPENDIX 2 



OUTLINE 

1. Organization and/or sources of information 

2. Tide and description of the management development program 

a. Objectives 

b. Participants— number, level, type 

c. Setting 

d. Duration and schedule 

e. Trainers 

f. Contents, methods and formats 

g. Remarks, if any 

3. Evaluation of the program 

a. Type of evaluation (reaction, learning, behavior change, results) 

b. Methodology 

c. Results 

d. Comments, if any 

4. Conclusions 
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